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ABSTRACT 


This  report  describes  an  adaptive  state  estimator  that  can  significantly 
improve  the  passive  range  and  depth  determination  of  a  randomly  maneu¬ 
vering  target.  The  target  in  this  study  is  a  submarine,  which,  while  being 
tracked,  performs  large-magnitude  depth  changes  at  times  unknown  to  the 
tracking  submarine. 

Present  passive  tracking  techniques  usually  utilize  a  Kalman  filter  to 
process  the  azimuth  and/or  elevation  observations.  A  Kalman  filter  will 
theoretically  give  the  "best"  estimates  of  target  range,  depth,  and  velocity 
when  the  system  and  measurement  errors  can  be  modeled  as  Gaussian  proc¬ 
esses.  The  main  difficulty  in  using  a  Kalman  filterin  passive  tracking  appli¬ 
cations  is  that  large  bias  errors  invariably  develop  as  the  target  makes 
large  alterations  in  velocity  or  depth.  A  technique  for  including  a  feedback- 
type  learning  processor  in  conjunction  with  the  Kalman  filter  has  been  found 
to  greatly  reduce  bias  errors  produced  by  the  maneuvering  target.  This 
error  elimination  is  accomplished  with  a  negligible  increase  in  computer 
storage  and  a  small  increase  in  computation  time.  The  method  is  general  in 
nature  and  can  be  applied  to  other  types  of  tracking  situations  in  which  a 
target  randomly  undergoes  large-magnitude  changes  in  motion. 
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AN  ADAPTIVE  ESTIMATOR  FOR  PASSIVE  RANGE  AND  DEPTH 
DETERMINATION  OF  A  MANEUVERING  TARGET 


INTRODUCTION 

The  usual  mathematical  analysis  of  a  tracking  situation  consists  of 
describing  the  target  dynamics  by  a  set  of  state -variable  equations  that  are 
driven  by  a  zero  (or  known)  mean  Gaussian  noise  process.  Knowledge  of  the 
state  variable  x  would  then  enable  one  to  compute  the  position,  velocity, 
course,  etc.,  of  the  moving  target.  Unfortunately,  not  all  state  variables  are 
directly  observable,  and  those  that  are  usually  appear  in  the  argument  of  an  arc¬ 
tangent  function.  The  observation  process  z.(t),  which  is  corrupted  by  additive 
Gaussian  noise,  is  then  operated  upon  by  a  Kalman  filter  to  provide  the  best 
(minimum  mean  square)  estimate  of  the  state  variable  x(t).  From  the  estimate 
at  time  t^,  denoted  x^,  the  desired  target-motion  parameters  are  computed. 

In  addition,  x^  is  utilized  to  form  the  one-step  predicted  estimate  Xfc+j,  which 
facilitates  linearization  of  the  next  observation  z^^. 

This  type  of  analysis  works  very  well  until  the  target  in  question  makes  a 
significant  alteration  of  course,  speed,  or,  in  this  specific  case,  depth.  When 
this  occurs,  the  depth  estimate  of  the  Kalman  filter  becomes  very  inaccurate  be¬ 
cause  a  large  bias  error  develops  from  faulty  knowledge  of  the  new  mean  value 
or  (unknown)  deterministic  system  input.  To  illustrate  the  magnitude  of  this 
error,  consider  the  computer -generated  curves  shown  in  figure  1.  The  heavier 
curve  represents  the  actual  (unknown)  relative  target  depth  plotted  versus  time; 
the  lighter  curve  is  the  depth  estimate  of  the  Kalman  filter  computed  from  a 
measurement  of  elevation  angle.  At  time  k  equal  200,  the  target  transits  to  a 
new  mean  depth  that  is  unknown  to  the  tracking  submarine .  The  magnitude  of 
resultant  steady-state  bias  error  in  the  estimate  is  readily  apparent,  and  it  is 
precisely  this  error  that  has  been  largely  eliminated.  This  is  demonstrated  by 
figure  2,  which  shows  the  response  of  the  adaptive  state  estimator  operating 
under  identical  conditions. 

To  model  the  large  variations  in  depth  to  which  a  modern  submarine  can 
decend,  certain  discrete  depths  (states)  dj,  d2,  ...,  dn  are  chosen.  These  n 
states  describe  in  discrete  terms  the  continuum  of  an  infinite  number  of  possible 
states.  It  was  found  that,  due  to  the  pull-in  feature  of  the  adaptive  filter,  n 
need  only  be  approximately  four  for  a  conventionally  powered  submarine  to 
provide  accurate  depth  estimation  at  any  target  depth. 
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Figure  1.  Actual  and  Estimated  Depths  versus  Time  for  Conventional  (Kalman)  Filter 
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To  statistically  model  the  maneuvering  target,  it  will  be  assumed  that 
the  n  states  can  be  described  by  a  semi-Markovian  process.  A  semi- 
Markovian  process  is  a  probabilistic  system  that  makes  its  state  transitions 
according  to  the  probability  matrix  of  a  Markovian  process.  2  However,  the 
time  spent  in  state  i  before  the  next  transition  to  state  k  is  a  random  variable 
governed  by  density  function  hj^(r).  By  incorporating  the  semi-Markovian  con¬ 
cept  into  a  Bayesian  estimation  scheme,  an  adaptive  state  estimator  was  de¬ 
veloped  that  can  handle  the  maneuvering  target  problem. 

Computer  simulations  have  been  accomplished  with  elevation-angle  obser¬ 
vations  only.  It  is  recognized  that  this  is  an  inadequate  approach  to  the  complete 
tracking-system  problem,  but  it  does  clearly  illustrate  the  adaptive  technique 
described  in  this  study.  To  show  development  of  the  adaptive  filter,  a  brief  re¬ 
view  of  Kalman  filtering  theory  is  presented.  Development  of  the  processor 
equations  is  then  followed  by  a  description  of  the  system  simulation  and  the 
results  obtained.  An  introduction  to  the  theory  of  semi-Markovian  processes 
is  presented  in  the  appendix. 


KALMAN  FILTERING  THEORY 

The  basic  technique  concerns  the  optimal  estimation  of  a  state  vector  x(t) 
from  a  noisy  set  of  observation  vectors  z(t)  that  is  related  to  x(t)  in  some 
functional  manner.  For  example,  the  state  vector  x  might  represent  the  posi¬ 
tion  and  velocity  of  a  satellite  and  the  observation  vector  z  the  slant  range  from 
a  ground  radar  site. 

In  1960,  Kalman  and  then  Bucy  presented  a  new  approach  to  the  state  esti¬ 
mation  problem.2* ^  Their  approach  consisted  of  modeling  all  random  processes 
by  state  equations  and  then  working  with  covariance  matrices  rather  than  corre¬ 
lation  functions.  The  reason  for  adopting  this  approach  was  to  ease  the  problem 
of  implementation  for  online  operation  of  a  digital  computer.  The  Kalman- Bucy 
filter  is  developed  from  a  pair  of  differential  equations  instead  of  the  much  more 
difficult  Weiner-Hopf  integral  equation  of  the  past;  it  also  provides  a  solution  to 
a  much  broader  class  of  filtering  and  estimation  problems  than  could  previously 
be  solved. 

Since  1961  there  have  been  many  alternative  derivations  of  the  basic  Kalman 
filter.  The  Bayesian  approach  developed  by  Ho  and  Lee2  is,  perhaps,  the  easi¬ 
est  to  understand.  However,  because  of  the  length  of  the  derivation,  only  the  re¬ 
sults  will  be  presented.  The  set  of  linear,  discrete,  time-varying  state  equations 
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representing  the  stochastic  process  (in  this  application,  target  motion)  is  given 
by  system  model. 

*k+l  =  <*>*k+1» +  r(k+1»  k)wk  +  \Hk+l,  k)uk  (1) 

and  observation  model 

%+i  =  H(k+1)2k+i+Vi- 

where  wk  and  vk+j  represent  white  Gaussian  noise  and  uk  is  a  deterministic 
input  that,  when  summed  with  wk,  yields  a  quantity  that  has  a  nonzero  mean 
Gaussian  distribution.  In  this  report  random  processes  wk  and  yk+j  will  be 
considered  uncorrelated  because  it  has  been  shown  by  many  authors  that  augment¬ 
ing  the  state  vector  x(t)  by  the  correlated  noise  process  yields  a  new  (larger 
dimensioned)  set  of  system  and  observation  equations  that  can  be  written  with  un¬ 
correlated  inputs.®  Notice  that  the  observation  process  is  a  linear  one,  and,  in 
addition,  the  following  a  priori  probabilistic  knowledge  is  known  about  the  inputs 
w  and  v: 


To  obtain  the  optimal  estimate  xk+1  for  state  vector  xk+1  (given  by  equa¬ 
tion  (1)),  the  criterion  for  optimization  will  be  minimization  of  mean-square 
error.  This  results  in  the  conditional  mean® 


min-  [E(4  -  4)T  <4  -  ik»]  =  E[ikl  h]  '  V  <3> 

The  conditional  mean  xj^  E[xk|Zk]  is  defined  as  the  expected  value  of 
state  vector  x  given  the  data  sequence  Zk  defined  by  .  z2 ,  . . . ,  z.k. 
Practically  speaking,  the  choice  of  minimization  of  mean-square  error  is  not 
nearly  as  restrictive  as  it  sounds.  It  is  shown  by  Meditch  and  others  that  for 
any  symmetric,  convex-upward  cost  function  the  best  estimator  is  always  the 
conditional  mean. 
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With  this  background  in  mind,  when  the  system  and  environment  are  known 
exactly  the  design  equations  for  the  optimal  estimator  can  be  expressed  as 

Vi "  ’ *K  + + “k+i  -  H*4  -  H*k>  <4> 

with  the  auxiliary  equation  defined  by 

Kalman  gain  matrix  Kk+J  =  +  R)_1 

M,  „  =  4>P  4>T  +  TQ  rT 
k+1  k 


Error  covariance  matrix  P.  , 

k+1 


(I  -  Kk+iH>\+1- 


(5) 


where 


H  =  H(k+1), 

4>  =  4>(k+l,k), 

R  =  R(k+1), 

Q  =  Q(k), 

r  =  r(k+i,k). 

An  examination  of  equation  (4)  shows  that  the  state  estimator  x^+j  can  be 
recursively  computed  by  updating  the  previous  state  estimate  with  a  simple 
summing  and  multiplying  operation.  In  order  to  compute  is  first 

calculated  from  the  previous  value  of  error  covariance  matrix  P^;  Kjr+i  is  then 
computed,  used  in  solving  equation  (4),  and  utilized  again  in  updating  Pj^+j  for 
the  beginning  of  the  next  cycle. 

Figure  3  illustrates  the  entire  Kalman  filter.  Note  that  the  Kalman  gain 
matrix  Kj^+i  is  time  varying  and  eventually  reaches  steady  state  only  for 
linear  time -invariant  systems. 
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Figure  3.  Block  Diagram  of  Kalman  Filter 
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DERIVATION  OF  ADAPTIVE  ESTIMATOR 


Consider  a  target  maneuvering  in  depth.  It  will  have  a  continuous  range 
in  which  to  choose  its  next  depth;  however,  because  a  submarine  is  approxi¬ 
mately  50  ft  from  keel  to  sail  top  and  since  the  adaptive  filter  has  a  strong  "pull 
in"  power,  only  a  smallnumber  n  need  actually  be  chosen.  Next,  the  n  possible 

depth  increments  are  defined  as  states  dj,  d£ . dn,  and  the  time  spent  in 

state  dj  before  the  next  transition  to  state  dj  is  assumed  to  be  a  random  vari¬ 
able  (r^j)  that  is  governed  by  a  time  density  function  of  the  form  hjj(r)  =  Ce“CT . 
In  addition,  the  probability  pjj  is  defined  as  the  probability  that  the  next  tran¬ 
sition  is  to  state  j  given  that  the  last  transition  was  to  state  i.  A  good  choice 
would  be  equally  probable  (1/n),  or  p^j  could  depend  upon  the  tactical  situation 
in  which  the  target  and  tracking  submarine  were  involved. 

Inasmuch  as  the  techniques  involved  are  general,  the  states  of  the  target 

are  assumed  to  be  sj,  S£ . sn,  where  —  depending  on  the  type  of  analysis 

desired  —  sj  could  represent  an  azimuth  increment,  depth  increment,  or  depth 
and  relative-target-velocity  pair.  To  derive  the  adaptive  estimator,  the  discrete 
time  form  of  the  condition  mean  is 


(6) 


where  =  Zj,  Z£,  ...,  z^  represents  the  data  sequence.  Defining  p(sjc  = 
si  |Zk)  as  the  probability  that  the  system  (target)  is  in  state  s^  at  time  given 
the  data  sequence  Z^,  the  second  term  in  equation  (6)  can  be  expressed 


n 


(7) 


i=l 

Combining  equations  (6)  and  (7)  yields 


n 


-k*  k 


’sk  =  8i>  p(sk*sil5k>’ 
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and  then  interchanging  the  order  of  summation  gives 


(8a) 


where 


4<VcX)v<*k|vsk=si> 


(8b) 


Equation  (8b)  represents  a  conventional  Kalman  filter  whose  internal  param¬ 
eters  are  conditioned  on  the  target  being  in  state  s^.  The  form  of  the  adaptive 
filter  given  by  equation  (8a)  is  a  bank  of  n  Kalman  filters,  each  multiplied  by  a 
time-varying  weighting  function  of  yet  undetermined  form.  However,  when  cer¬ 
tain  practical  assumptions  are  made,  the  form  of  the  adaptive  filter  is  greatly 
simplified. 


The  basic  form  was  first  developed  by  Magill^  to  solve  the  problem  of  the 
unknown,  nonvarying  stochastic  system.  It  next  appeared  in  the  solution  of  the 
Markovian  switching -environment  problem  as  formulated  by  Ackerson  and  Fu,  ® 
and  later  in  the  random-switching  stochastic  system  problem  of  Moose  and 


Wang. £  Upon  cursory  examination,  equation  (8a)  would  appear  to  be  essentially 


— -o-  -r - ^  - '  '  '  “  « 

the  same  as  previously  reported  (by  Magill  and  by  Fu);  however,  there  is  a  major 


r  J  r  \  J  Z>  J  /*  »  J 

difference,  and  it  lies  in  the  computation  of  the  time -varying  weighting  functions 

* 


P(Sk  =  si|?k). 


a* 


To  recursively  compute  the  a  posteriori  probabilities  P(Sjj+1  =  si  |?k+l)» 
the  computational  sequence  is  as  follows:  first  the  previously  stored  value 

= 


p(sk  “  s^l  Zjj)  is  updated  by  a  semi-Markovian  prediction  process  to  p(sjc+j 
Si  (Z^);  then  a  new  measurement  is  taken,  and  p(sjc+1  =  SijZjj+j)  is  computed, 
used  in  the  adaptive  filter,  and  stored  to  begin  the  next  cycle. 


The  preceding  qualitative  procedure  can  be  expressed  in  mathematical 
terms  by  a  set  of  equations  in  which  data  sequence  4  (Z^,  z^^)  is  utilized. 

By  Bayes  rule, 


p(s 


k+1 


W 


p(2k+ll\+l”si'?k)p<Vl  =  si|4> 


P<4+1 


4> 


(9) 
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It  has  been  pointed  out  that  the  denominator  is  common  for  all  weighting  terms 
and  can  be  replaced  by  a  constant.  Therefore  only  the  nuiperator  is  of  interest . 

It  is  possible  to  show  that  p(Z|c+^  |  sk+l  =  si*  %k)  *s  approximately  normally  dis¬ 
tributed  and  can  be  represented  by  the  known  Gaussian  density  function  established 
from  the  Kalman  filtering  algorithms  conditioned  on  sj  —  that  is,  N  |H4>(Si)x^  (sj), 
[HMk+i(Si)HT  +  R]|,  which  is  just  a  number  when  evaluated.  Note  that 
the  term  p(sk+i  =  sj  JZ^)  in  expression  (9)  is  the  predicted  value  that  will  be 
generated  by  the  semi-Markovian  process.  Expanding  P(Sk+i  =  Sj  |  Z^)  yields 


n 


a=l 


The  first  term  of  equation  (10)  is  conditioned  on  both  =  sa  and  data  se¬ 
quence  Zjj.  It  has  been  established  that  Z^  and  p(s^  =  sa)  are  strongly  de¬ 
pendent;  in  fact,  Zk  actually  aids  in  determining  p(sjc  =  sa),  but,  inasmuch  as 
this  is  already  given,  P(Sk+i  =  si|sk  =  sa»  -\J  can  be  expressed  as  p(Sk+i  = 
si|sk  =  sa).  The  second  term  p(sk  =  sa|Zji)  is  known  from  the  previous  re¬ 
cursive  calculation.  Combining  equations  (9)  and  (10)  yields 


p(s 


k+1 


=  Si|^k+l)=  (CONST) 


n 


E 


p(s 


k+1 


(11) 


The  only  term  undefined  in  equation  (11)  is  p(Sk+j  =  Sjls^  =  sa);  the  implication 
of  this  conditional  probability  can  be  expressed  as  PR  [system  is  in  state  (i)  at 
time  tk+j  |  system  is  in  state  (a)  at  time  t^],  but  this  is  exactly  the  same  defi¬ 
nition  given  in  the  appendix  as  the  "random  starting"  probability  ^aj(t^+1  -  t^). 
Therefore  equation  (1 0)  can  be  expressed  in  its  final  form  as 

r  n 


p(s 


k+1 


=  s1|Zk+1)=(C°NST) 


w 


^ai(tk+l  “  tk)  * 


(12) 


Notice  that  only  n  terms  need  be  stored  from  sample  to  sample. 
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For  uniform  sampling,  the  case  of  primary  concern,  0ai(t^+1  -  t^)  = 

0a i  [(k+l)T  -  kT]  =  0ai(T),  which  depends  only  upon  the  sample  spacing  T.  A 
good  engineering  approximation  is  to  let  0^  =  0.  95  and  0y  =  0.  05/n-l  for  i  /  j. 

One  other  item  needed  in  the  computation  of  p(sjc+j  =  si|?k+i)  *s 
initial  probability  of  being  in  state  s^  at  time  zero.  It  was  found  that  the  adap¬ 
tive  filter  was  relatively  insensitive  to  the  choice  of  initial  probabilities.  Inas¬ 
much  as  the  adaptive  filter  rapidly  learns  the  true  system  configuration,  as  data 
are  observed,  one  might  choose  equally  probable  a  priori  estimates  with  little 
degradation  in  performance. 

To  simplify  the  optimal  adaptive  estimator  given  as  equation  (8a),  assume 
that  the  R(sj)  and  Q(sj)  covariance  matrices  are  identical.  This  assumption 
is  not  overly  restrictive  inasmuch  as  the  mean  (u^),  which  is  of  primary  con¬ 
cern,  can  be  any  value.  With  this  assumption,  the  Kalman  gain  matrices  be¬ 
come  identical  for  all  n;  and  it  then  follows  from  equations  (1)  and  (8a)  that  the 
adaptive  estimator  can  be  expressed  in  its  final  form  as  the  simplified  adaptive 
estimator 


Kn  =«VrV  VifVi  -  H4V  HraJ 


where 

n 


(13) 


i=l 


and  P(Sk+i  =  Si|Zk+i)  is  given  by  equation  (12)  and  K^+j  =  Kalman  gain 
matrix  equation  (5). 

Equation  (13)  has  the  form  of  a  single  Kalman  filter  with  an  external  learn¬ 
ing  feature  u^,  which  is  the  estimated  (unknown)  deterministic  target  depth  com¬ 
mand.  Figure  4  shows  the  form  of  the  adaptive  filter  to  estimate  the  range  and 
depth  of  a  maneuvering  target.  It  is  this  form  of  the  adaptive  filter  that  will  be 
used  in  the  computer  simulations  that  follow. 
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Figure  4.  Block  Diagram  of  Adaptive  Filter 


SIMULATION  AND  RESULTS 


To  approximate  target  motion  in  the  xy  plane,  a  set  of  two  second-order 
differential  equations  is  written.  The  system  equations  are  next  converted  to  a 
set  of  four  state  equations,  sampled  at  uniform  discrete  intervals,  and  then 
transformed  to  the  familiar  matrix  difference  equation  form 


-k+1 


=*sk  +  n»k  +uk) . 


The  vector  u^  contains  the  unknown  deterministic  (random  to  the  filter)  sequence 
of  depth  commands,  and,  in  addition,  an  input  is  generated  to  make  the  relative 
target  range  slowly  increase  with  time.  The  observation  process  is  of  the  form 
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where  the  magnitude  of  elevation-angle  error  is  chosen  to  represent  meas¬ 

ured  data  of  a  present-day  Fleet  sonar  system.  The  nonlinear  observation  pro¬ 
cess  is  next  linearized  by  the  series  expansion  of 


around  the  predicted  value  of  the  previous  estimate  x^  as  shown  in  figure  5.  By 
so  doing,  the  observation  process  is  converted  to  a  linear  time -varying  measure¬ 
ment  vector  containing  state  variables  xj  and  x3  representing,  respectively,  x 
and  y. 

In  figure  5,  the  data-generation  process  is  shown  to  the  left  of  the  broken 
line,  and  to  the  right  the  adaptive  filter  is  within  the  dashed  line  with  the  linear¬ 
ization  process  directly  below. 

In  designing  the  adaptive  filter,  four  depths  dj,  d3,  d3,  and  d4  are 
chosen.  To  model  the  inaccurate  relative  target  velocity  initially  estimated 
from  external  sources,  a  pair  of  relative  opening  velocities  vrj,vr2  is  chosen. 
This  leads  to  a  selection  of  n  =  8  states  where  each  state  S[  is  the  pair  (d^, 

Vri).  The  increase  in  computation  time  over  that  of  a  conventional  Kalman  filter 
was  approximately  37  percent,  which  was  determined  by  noting  that  the  compu¬ 
tation  time  to  perform  400  complete  cycles  on  the  UNIVAC  1108  was  increased 
from  8  to  11  seconds  upon  changing  from  the  Kalman  to  the  adaptive  filter. 

Numerous  computer  runs  were  made  to  compare  the  performance  of  range 
and  depth  estimation  of  the  conventional  (Kalman)  filter  with  that  of  the  adaptive 
filter.  Inasmuch  as  elevation  angle  was  the  only  parameter  measured,  a  good 
estimate  of  initial  range  was  assumed.  Figures  6  and  7  each  show  the  unknown 
true  target  depth  (heavier  curve)  and  the  estimated  depth  (lighter  curve)  plotted 
versus  time  for  conventional  and  adaptive  filters,  respectively.  An  initial 
depth  error  was  assumed  at  k  =  0,  and  at  unknown  times  k  =  74  and  k  =  280 
the  target  underwent  major  depth  changes.  A  comparison  of  figures  6  and  7 
shows  that  the  large  bias  error  is  eliminated  by  the  adaptive  filter. 

The  curves  shown  in  figures  8  and  9  illustrate  target  range  versus  time 
for  the  conventional  and  adaptive  filters,  respectively.  It  is  seen  that  the  mean- 
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Figure  7.  Actual  and  Estimated  Depths  versus  Time  for  Adaptive  Estimator 
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Figure  8.  Actual  and  Estimated  Ranges  versus  Time  for  Conventional  Filter 
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Figure  9.  Actual  and  Estimated  Ranges  versus  Time  for  Adaptive  Estimator 
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square  error  of  (r  -  r)2  steadily  increases  with  increasing  time  with  the  con¬ 
ventional  filter  but  remains  fairly  constant  with  the  adaptive  filter.  This  set  of 
curves  illustrates  the  effect  of  a  faulty  knowledge  of  relative  target  velocity  of 
the  same  degree  of  accuracy  as  that  of  present  Fleet  sonar  systems. 


CONCLUSIONS 

It  now  appears  feasible  to  statistically  model  a  maneuvering  target  as  a 
semi-Markovian  process  and  then  incorporate  the  statistics  into  the  design  of 
an  adaptive  state  estimator  that  consists  of  a  Kalman  filter  with  a  learning 
algorithm  connected  in  a  feedback  manner.  The  performance  of  the  adaptive 
estimator  was  compared  with  that  of  a  conventional  Kalman  filter  in  estimating 
the  range  and  depth  of  a  maneuvering  target  by  using  only  measurements  of  ele¬ 
vation  angle.  The  simulations  show  that  once  a  target  makes  a  significant  alter¬ 
ation  of  depth  or  speed  the  Kalman  filter  develops  a  significant  bias  error  that 
the  adaptive  estimator  largely  eliminates.  Bias  in  the  Kalman  filter  has  been 
shown  to  arise  from  imperfect  knowledge  of  the  deterministic  control  vector  ujj, 
which  acts  as  a  mean  vector  when  summed  with  the  Gaussian  white-noise  system 
input  wk. 

The  superior  performance  of  the  adaptive  filter  was  gained  at  the  expense 
of  increased  computation  time  —  approximately  35  percent  greater  than  that  of 
the  Kalman  filter.  This  disadvantage  is  minor  if  a  good  estimate  is  desired 
rather  than  a  bad  one .  Although  range  and  depth  were  the  target  parameters  of 
interest,  the  processor  equations  were  developed  in  such  a  manner  that  they 
would  apply  to  other  tracking  situations  as  well. 
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Appendix 

SEMI-MARKOVIAN  PROCESSES 


To  fully  understand  the  adaptive  state  estimator,  one  must  have  a  basic 
understanding  of  semi-Markovian  statistics  and  the  manner  in  which  they  apply 
to  large-scale  dynamic  systems  containing  random  switching  parameters.  The 
distinguishing  feature  of  a  semi-Markovian  process  is  that  the  time  of  transition 
between  states  is  a  random  variable,  and  the  process  is  Markovian  only  at  the  ac¬ 
tual  time  of  transition.  This  type  of  process  is  very  general  in  nature  and  contains 
as  special  cases  the  discrete  and  continuous -time  Markovian  processes. 

Much  of  the  summary  concerning  the  semi-Markovian  process  presented 
in  this  appendix  was  taken  from  a  very  fine  paper  by  R.  A.  Howard  that  ap¬ 
peared  in  the  Institute  of  Electrical  and  Electronics  Engineers  Transactions  on 
Military  Electronics  (reference  1  of  the  basic  text).  It  is  precisely  because 
semi-Markovian  statistics  are  so  little  known,  but  potentially  such  a  powerful 
tool,  that  this  appendix  is  written. 


DEFINITION  OF  TERMS 

A  semi-Markovian  process  is  a  probabilistic  system  that  makes  its  state 
(parameter  vector  or  system  configuration)  transitions  according  to  the  transi¬ 
tion  probability  matrix  P  of  a  Markovian  process.  However,  the  time  spent  in 
state  i  before  the  next  transition  to  state  j  is  a  random  variable,  and  it  is 
this  property  that  distinguishes  a  semi-Markovian  process  from  a  Markovian. 

The  objective  is  to  calculate  0ij(t-to)  4  PR  [that  a  system  is  in  state  j  at  time  t, 
given  that  the  system  entered  state  i  at  time  zero],  (For  the  very  important 
practical  case  of  exponential  density  functions  hy(-r),  the  conditional  probabili¬ 
ties  0jj(T)  can  be  simply  computed  and  then  directly  incorporated  into  the  adap¬ 
tive  filter. ) 

In  order  to  do  so,  begin  by  establishing  the  following  set  of  definitions. 

Let  pij  be  the  conditional  probability  that  the  next  transition  is  to  state  j, 
given  that  the  last  transition  was  to  state  i.  Define  Tjj  as  a  random  variable, 
the  time  spent  in  state  i,  given  that  the  next  transition  is  to  j;  then  let  hjj(r) 
be  the  probability  density  function  governing  Ty.  Therefore,  to  completely 
specify  the  semi-Markovian  process  for  an  n  state  switching  system,  n2  elements 
Pjj  and  n2  elements  hy(T)  are  needed  and  must  satisfy  the  following  conditions: 
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n 


j=l 


oo 


J  hyty)  dT  =  1 


(A-l) 


0 


With  this  a  priori  probabilistic  knowledge  concerning  the  switching  system, 
suppose  that  the  process  enters  state  i  and  chooses  as  its  successor  state  j, 
but  that  an  observer  does  not  know  the  successor  chosen.  The  best  one  can  do  to 
make  an  estimate  of  the  time  t}  (time  spent  in  state  i)  is  to  calculate  the 
weighted  average  of  the  holding- time  density  functions  h^j (t).  Defining  Wj(t)  as 
the  waiting-time  density  function  governing  tj*  this  can  be  expressed 


n 


(A-2) 


j=l 


DERIVATION  OF  CONDITIONAL  STATE  PROBABILITIES 


To  proceed  with  determination  of  the  conditional  state  probabilities  for  a 
semi-Markovian  process,  define  0ij(t)  as  the  conditional  probability  that  the 
system  is  in  state  j  at  time  t,  given  that  the  system  entered  state  i  at  time 
zero.  There  are  two  mutually  exclusive  ways  for  0y(t)  to  occur. 

Consider  the  first  case,  in  which  i  =  j  and  the  system  has  remained  in 
state  i  for  tj  >  t.  This  can  be  expressed 


Case  I  = 
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Figure  A-l.  Diagram  of  Possible  State  Transitions  for  Case  n 

where  6jj,  the  Kronecker  delta  function,  ensures  that  the  preceding  term  only 
exists  for  i  =  j. 

The  second  manner,  case  n,  in  which  0ij(t)  can  occur  is  for  the  system 
to  leave  i  at  time  t,  go  to  some  intermediate  state  k,  possibly  itself,  then 
eventually  proceed  from  state  k  to  final  state  j  in  the  remaining  time  (t  -  t). 
As  figure  A-l  shows,  these  two  happenings  are  independent  of  each  other,  and 
therefore  the  probabilities  multiply.  Summing  over  all  Intermediate  states  k 
and  Integrating  over  all  possible  time  t  spent  in  state  i  yields,  for  case  n, 


t 


n 


(A -4) 


When  equations  (A-3)  and  (A-4)  are  combined,  the  final  expression  for  the  con¬ 
ditional  state  probabilities  is  given  by 


t  \  n  t 


-  j drj  J,.  +^Plfc  J  hlk(T)  8y(t-r)  dr.  (A-5) 


hik  ^  0kj  dT*  <A~5) 


0  /  k=l  o 


Equation  (A-5)  is  an  integral  equation  in  0jj.  However,  it  is  easily  solved  be- 

f  Un  lnnt  f/vWM  WAnWAMAntn  n  AAmmltlH  An  tn  f  Ka  flmA  /IataaIa  ^Ua+  Kh  aaaaa  a  a 


cause  the  last  term  represents  a  convolution  in  the  time  domain  that  becomes  a 
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product  in  the  complex  frequency  domain.  Hence,  upon  taking  the  Laplace 
transform, 


-  wt(s)) 


V 


•X>ikVs>  vb> 


k=l 


(A-6) 


The  algebraic  expression  given  by  equation  (A-6)  is  still  a  very  cumber¬ 
some  form  to  manipulate.  It  is  possible,  however,  to  express  the  results  in  a 
matrix  manner  by  defining  a  very  special  form  of  matrix  multiplication  with  the 
symbol  denoted  by  »  .  Upon  expanding  equation  (A-6)  term  by  term  and  re¬ 
grouping  like  elements,  the  following  expression  can  be  developed: 


1  0  ...  O' 
0  1  ...  0 

•  •  • 

Lo  ...  lj 


pnhn  ...  Plnhln 
P21h21  P2nh2n 


•••  Pnn^n  , 
ni  nl  nn  nn 


*11  *12  *•*  *ln 
*21  *•*  *2n 


*nl  **•  *nn- 


1_ 

8 


(1-Wj)  0 


(l-w2) 


(1-w  ) 
'  n  - 


(A-7) 


where  all  matrices  are  of  dimension  n  x  n.  Now  pause  and  examine  the  second 
term  on  the  left  side  of  equation  A-7 .  Each  term  cy  is  equal  to  pyh^.(s); 
therefore,  define  the  special  matrix  multiplication  C  A  A  »  B,  whereby  each 
element  cy  4  aij^ij  •  With  this  convention,  I  being  the  identity  matrix,  equa¬ 
tion  A-7  can  be  expressed 

[I-Po  H(s)]  0(s)=^[l- W(s)]  , 


which  yields  the  basic  transform  pair 
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0(s)4[I-P»H(s)]'1  [I-W(s)] 

S 

e(t)  =  l"1  [e(s>] . 


(A-8) 


o 

The  matrix  0(t)  has  n  elements  0.i(t),  which  are  exactly  the  condition 


al  state  probabilities  that  were  to  be  determined.  They  result  when  the  Inverse 
Laplace  transform  of  each  element  of  the  matrix  expression  0(s)  is  taken.  In 
order  to  Illustrate  equation  (A-8 )  and  to  derive  some  meaningful  results,  an  ex¬ 
ample  is  presented  that  Includes  all  of  the  previous  design  techniques. 


EXAMPLE  OF  A  CONTINUOUS  TIME  DENSITY  FUNCTION 

An  important  example,  the  results  of  which  will  later  be  incorporated  into 
the  simulation  of  the  adaptive  filter,  is  the  following:  Consider  n  possible 
states  (system  configurations),  each  having  Identical  exponential  density  func¬ 
tions  hjj(t)  =  ae-at  and  an  arbitrary  transition  matrix  P  containing  n2  ele¬ 
ments  pfj.  It  is  desired  to  determine  the  n2  conditional  state  probabilities 
0jj(t).  From  equation  (A-2),  for  wj(s), 


n 


oe 


0 


With  this  information  the  terms  needed  in  the  basic  design  equation  (A-8)  can  be 
computed: 


(A-10) 


I  4 
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Upon  combining  equations  (A-8),  (A-10),  and  (A-ll),  the  transform  pair  of  the 
conditional  state  probability  matrices  can  be  expressed  as 


0(s)  =  [si  +  a(I  -  P)] 

e(t)  =  e-<I-p)at.  <A-12» 


Equation  (A-12)  shows  that  the  case  of  identically  distributed,  exponential  den¬ 
sity  functions  leads  to  the  important  result  that  0(t)  =  exp  (At),  where  the 
matrix  A  is  given  by  the  known  quantify  -a (I  -  P). 
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